A Comparative Study of Sentiment Analysis on Social Media Tweets Using Traditional Machine Learning and Deep Learning Approaches

Abstract

Social media platforms, particularly Twitter (now X), generate enormous volumes of user-generated textual content every second, making automated opinion mining an indispensable tool for researchers, businesses, and policy makers. This paper presents a comparative investigation of sentiment classification techniques applied to tweet datasets, spanning classical probabilistic methods such as Naïve Bayes and Support Vector Machines (SVM), lexicon-based approaches including VADER, and contemporary deep learning architectures centred on fine-tuned BERT transformers. A systematic pre-processing pipeline—comprising tokenisation, stop-word removal, URL stripping, and stemming—is applied uniformly across all models to ensure fair comparison. Experimental evaluation on the publicly available Sanders Twitter corpus (5,512 annotated tweets) reveals that a hybrid Naïve Bayes model enriched with a domain lexicon achieves 88.4% accuracy, while fine-tuned BERT surpasses all baselines at 91.2% accuracy, 90.8% precision, and 90.6% F1-score. The study further identifies key challenges including slang interpretation, sarcasm detection, and class imbalance, and suggests a path toward lightweight, hybrid architectures suitable for real-time deployment in resource-constrained environments.

Introduction

This study examines sentiment analysis on Twitter, a major micro-blogging platform with over 206 million daily active users, making it a valuable source of opinion data on topics such as politics, products, public health, and social issues. Sentiment analysis aims to classify text as positive, negative, or neutral and has evolved from simple rule-based methods to advanced machine learning and deep learning techniques.

Background and Motivation

Early sentiment analysis relied on lexicon-based approaches that assigned sentiment scores to words. However, these methods struggled with informal social media language, including abbreviations, emojis, and sarcasm. Machine learning models such as Naïve Bayes (NB) and Support Vector Machines (SVM) improved performance by learning from labeled data, while word embeddings like Word2Vec and GloVe enhanced contextual understanding. More recently, transformer-based models such as BERT have achieved state-of-the-art results by capturing deep contextual relationships in text.

The study addresses a research gap by providing a systematic comparison of lexicon-based, classical machine learning, hybrid, and deep learning approaches under a unified preprocessing framework.

Literature Review

Research in sentiment analysis has progressed through three generations:

1. Lexicon-based methods (e.g., VADER) that rely on sentiment dictionaries and rules.
2. Feature-engineered machine learning models such as NB and SVM.
3. Transformer-based models like BERT and RoBERTa.

Previous studies consistently show that machine learning models outperform lexicon-based methods when labeled data is available, while BERT-based models generally achieve the highest accuracy. However, these advanced models require greater computational resources.

Dataset and Preprocessing

The study uses the Sanders Twitter Sentiment Corpus, containing 5,512 manually labeled tweets related to Apple, Google, Microsoft, and Twitter. Tweets are classified into four categories:

• Positive (10.3%)
• Negative (11.9%)
• Neutral (45.4%)
• Irrelevant (32.4%)

A comprehensive preprocessing pipeline was applied, including:

• Removal of URLs and user mentions
• Hashtag normalization
• Emoji and emoticon conversion
• Lowercasing and punctuation removal
• Stop-word removal and stemming
• Tokenization

Tweets were represented using Bag-of-Words, TF-IDF vectors, or contextual embeddings depending on the model.

Sentiment Analysis Techniques Evaluated

The study compares seven approaches:

• Naïve Bayes (NB) – probabilistic classifier for text data.
• Support Vector Machine (SVM) – maximum-margin classifier using TF-IDF features.
• Logistic Regression – interpretable linear classification model.
• Random Forest – ensemble learning method using multiple decision trees.
• VADER – social-media-specific lexicon-based sentiment analyzer.
• Hybrid NB + Lexicon – combines NB predictions with VADER sentiment scores.
• Fine-tuned BERT – transformer-based deep learning model trained on the dataset.

Results

Performance on binary sentiment classification (Positive vs. Negative) showed:

Key findings:

• BERT achieved the highest accuracy (91.2%) and F1-score (90.6%), confirming the effectiveness of transformer-based models.
• Hybrid NB + Lexicon significantly outperformed traditional machine learning models, demonstrating the value of combining lexicon knowledge with statistical learning.
• VADER, while computationally efficient and requiring no training data, showed the lowest performance.
• Classical machine learning models such as SVM and Random Forest remained competitive and computationally less expensive than deep learning models.

Conclusion

This paper presented a comprehensive comparative study of sentiment analysis techniques applied to Twitter data, evaluating VADER, Naïve Bayes, SVM, Logistic Regression, Random Forest, a Hybrid NB-Lexicon model, and fine-tuned BERT under a strictly consistent experimental protocol. The key finding is that while classical ML models offer a favourable accuracy-to-complexity trade-off—with the Hybrid NB model achieving 88.4%—fine-tuned BERT sets the performance ceiling at 91.2% accuracy and 90.6% F1-score by virtue of its deep contextual representations and large-scale pre-training. The study confirms that no single technique dominates across all deployment contexts. VADER suits resource-constrained real-time applications where no labelled training data is available. The Hybrid NB model provides a strong balance of accuracy and speed. BERT is preferred whenever accuracy is paramount and computational resources are accessible. This tiered model selection framework offers practitioners a principled guide for choosing the appropriate technique based on their operational constraints. Future research should (a) explore lightweight transformer distillations such as DistilBERT or TinyBERT to close the gap between accuracy and inference efficiency, (b) extend evaluation to multilingual and code-switched tweet corpora relevant to the linguistic context, (c) incorporate multi-modal signals including images, emoji semantics, and video captions to overcome the limitations of text-only analysis, and (d) develop continual learning frameworks that adapt to temporal drift in social media language without full retraining cycles.

References

[1] Y. Qi and Z. Shabrina, “Sentiment analysis using Twitter data: a comparative application of lexicon- and machine-learning-based approach,” Social Network Analysis and Mining, vol. 13, no. 1, p. 31, Feb. 2023. https://doi.org/10.1007/s13278-023-01030-x [2] S. Gharge and M. Chavan, “An Integrated Approach for Sentiment Tweets Detection using NLP,” in Proc. IEEE Int. Conf. on Electronics, Communication and Aerospace Technology (ICECA), 2017, pp. 1–4. [3] J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” in Proc. NAACL-HLT, 2019, pp. 4171–4186. [4] P. Turney, “Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews,” in Proc. 40th Annual Meeting of the ACL, 2002, pp. 417–424. [5] A. Go, R. Bhayani, and L. Huang, “Twitter sentiment classification using distant supervision,” CS224N Project Report, Stanford University, vol. 1, p. 12, 2009. [6] S. Saha, M. I. H. Showrov, M. M. Rahman, and M. Z. H. Majumder, “VADER vs. BERT: A Comparative Performance Analysis for Sentiment on Coronavirus Outbreak,” in Machine Intelligence and Emerging Technologies – MIET 2022, LNICST 490, Springer, 2023, pp. 371–385. https://doi.org/10.1007/978-3-031-34619-4_30 [7] P. Gamallo and M. Garcia, “A Naïve-Bayes Strategy for Sentiment Analysis on English Tweets,” in Proc. 8th Int. Workshop on Semantic Evaluation (SemEval 2014), pp. 171–175. [8] R. Ramezani, “Enhancing Sentiment Analysis Accuracy on Social Media Comments using a Tuned BERT Model,” Discover Computing, Springer Nature, 2025. https://doi.org/10.1007/s10791-025-09599-x [9] R. Bharadwaj, S. Shendurkar, T. Kadam, U. Patekar, and S. Waghule, “RoBERTa and VADER Sentiment Analysis: A Comparative Approach,” in Big Data and Computing Advances, Springer, 2024. https://doi.org/10.1007/978-981-97-8666-4_12 [10] Comprehensive NLP Review, “Sentiment Analysis of Twitter Data Using NLP Models: A Comprehensive Review,” ResearchGate, Mar. 2026. Available: https://www.researchgate.net/publication/388980597 [11] V. Narayanan, I. Arora, and A. Bhatia, “Fast and accurate sentiment classification using an enhanced Naive Bayes model,” in Intelligent Data Engineering and Automated Learning – IDEAL 2013, LNCS 8206, Springer, 2013, pp. 194–201. [12] L. Dey, S. Chakraborty, A. Biswas, B. Bose, and S. Tiwari, “Sentiment Analysis of Review Datasets Using Naïve Bayes and K-NN Classifier,” Int. Journal of Information Engineering and Electronic Business (IJIEEB), vol. 8, no. 4, pp. 54–62, 2016. [13] C. Chen, J. Zhang, Y. Xie, Y. Xiang, and W. Zhou, “A Performance Evaluation of Machine Learning-Based Streaming Spam Tweets Detection,” IEEE Transactions on Computational Social Systems, vol. 2, no. 3, pp. 65–76, Sep. 2015. [14] W. Medhat, A. Hassan, and H. Korashy, “Sentiment analysis algorithms and applications: A survey,” Ain Shams Engineering Journal, vol. 5, no. 4, pp. 1093–1113, 2014. [15] J. Serrano-Guerrero, J. A. Olivas, F. P. Romero, and E. Herrera-Viedma, “Sentiment analysis: A review and comparative analysis of web services,” Information Sciences, vol. 311, pp. 18–38, 2015.

Copyright

Copyright © 2026 Monica Dnyandeo Thete. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.